This invention relates to automatic face authentication and recognition. More particularly, it relates to a method and system for authentication and recognition of faces using a three-dimensional facial surface representation and facial bilateral symmetry plane extraction to derive a profile curve and a coordinate system for aligning facial representations for comparison.
Automatic face authentication refers to using facial images or scans to verify an identity claim of a known individual. Automatic face authentication has long been an active research area for its wide potential applications, such as law enforcement, security access, and man-machine interaction. Authentication involves performing verification based on a one-to-one search to validate the identity claim of an individual (i.e., access control for a building, room, or for making a transaction at an ATM terminal). Automatic face recognition refers to using facial images or scans to identify an unknown individual within a database of known individuals. Recognition in one-to-many searches is based on comparison to a database of known individuals (e.g., law enforcement, surveillance, and recently driver licenses). Face authentication is in one sense a simpler process than face recognition: comparisons are made only to the claimed identity, and a threshold of similarity is used to accept or reject the claim. In another sense, authentication is more difficult, because of the need to determine this threshold rather than using a “best match” criterion as in many face recognition applications. With face authentication, the group of invalid IDs (imposters) is, by definition, not in the reference database. Therefore, face authentication methods must successfully operate in 1-to-1 comparisons, without knowledge of possible errors in claims (i.e., who else might the individual be).
Several approaches have been promoted to recognize and authenticate an individual or a group of people. Access control applications authenticate by physical appearance (by guard personnel, receptionist); by something the individual knows (pins, passwords); by something the individual has (lock/key, card, badge, token); by biometric evidence (a unique physiological or behavioral characteristic of the individual); or by a combination of both “what one has” (i.e., a card) and “what one knows” (i.e., their passcode). Most workplace entry points are typically controlled by a badge/card or by physical appearance. All of these methods, except biometrics, are fallible and can be circumvented, lost, or stolen. Interest in authentication using biometrics is therefore growing dramatically.
Biometric access control uses measurable physiological or behavioral traits to automatically authenticate a person's identity. Biometric characteristics must be distinctive of an individual, easily acquired and measured, and comparable for purposes of security validation. The characteristic should change little over time (i.e., with age or voluntary change in appearance) and be difficult to change, circumvent, manipulate, or reproduce by other means. Typically, high-level computer based algorithms and database systems analyze the acquired biometric features and compare them to features known or enrolled in the database. The mainstream biometric technologies use morphological feature recognition such as fingerprints, hand geometry, iris and retina scanning, and two dimensional (2D) face authentication. Each of these except face authentication is either intrusive or fails in some cases (e.g., about 10% of population do not have good enough fingerprints).
There has been a large body of literature on 2D face recognition and authentication. For an overview, see R. Chellappa, C. Wilson, and S. Sirohey. Human and machine recognition of faces: A survey, Proceedings of the IEEE, 83(5):705-740 (1995). Among various approaches, Principal Components Analysis (PCA) to face imaging, popularly called eigenfaces, is now a cornerstone in face recognition. For a more detailed explanation of PCA, see Turk, M., Pentland, A, Face recognition using eigenfaces, Proc. CVPR, 1991, pp 586-591. 2D face authentication, though less intrusive than other biometric technologies, has simply not attained the degree of accuracy necessary in a security setting. 2D face recognition methods are in general unable to overcome the problems resulting from illumination, expression or pose variations, facial hair and orientation.
The emerging trend noted by many researchers in the field of face recognition is the 3D technology, which offers several additional advantages to 2D face recognition. 3D technology is expected to be more accurate and able to overcome the problems of 2D methods, because 3D information is viewpoint and lighting condition independent. There are several strategies in 3D face recognition. Some researchers try to segment the 3D face surface into meaningful physiological points, lines and regions based on curvature analysis at each point. For example, Hallinan et al. utilized curvature properties to segment a face surface into regions, and a set of twelve features were extracted for face recognition. P. Hallinan, G. G. Gorden, A. L. Yuille, et al., Two and three-dimensional patterns of the face, A. K. Peters (ed), A K Peters Ltd (1999). Moreno et al., used a HK segmentation (based on the analysis of signs of mean and Gaussian curvatures at each point) to isolate regions of pronounced curvature, and up to eighty-six descriptors were obtained from the segmented regions. A. B. Moreno, Á. Sánchez, J. F. Vélez, et al., Face recognition using 3D surface-extracted descriptors, Proceedings of the 7th Irish Machine Vision & Image Processing Conference, Antrim, N. Ireland, 2003. For the presence of noise, expression variance and incomplete scanning, however, curvature based feature extraction is not robust enough for face recognition. For example, Moreno et al. reported only a 78% correct recognition rate.
In some methods, 3D face modeling has been used as an enhancement of 2D analysis methods. For example, Blanz et al used a 3D morphable face model, which is learned from a set of textured 3D scans of heads, to encode images. V. Blanz, T. Vetter. Face Recognition Based on Fitting a 3D Morphable Model, IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9) (2003). Recognition is performed based on the model coefficients created in the process of fitting the morphable model to images. Lee et al. also presented a model-base face recognition approach under a similar framework. M. W. Lee, S. Ranganath, Pose-invariant face recognition using a 3D deformable model, Pattern Recognition, 36:1835-1846 (2003). In their method, the deformable 3D face model is a composite of an edge model, a color region model and a wireframe model. This strategy is a 2D solution in nature, for the media to be compared in these studies is still 2D intensity images. A problem with this strategy is that fitting the morphable model to images is a computational expensive process. As reported by Lee et al., 4.5 minutes are needed for fitting the model to an image on a workstation with a 2 GHz Pentium4 processor.
Chang et al. used both 2D and 3D face information for the recognition task. K. I. Chang, K. W. Bowyer, P. J. Flynn, Face Recognition Using 2D and 3D Facial Data, The Proceedings of Workshop in Multimodal User Authentication, pp. 25-32, Santa Barbara, Calif., USA (2003). In their experiments, a PCA-based approach was tuned for face recognition from 2D intensity images and 3D range images, respectively. Their comparison result is that 3D outperforms 2D. By combining the 2D distance metric and the 3D distance metric, a 2D-plus-3D criterion is used during the decision process of face recognition. In their experiments, pose variations that occur during the acquisition process are manually normalized. The recognition rate of the combination scheme was reported to be higher than 98% under the condition that the 2D and 3D images are taken in a front view and the subjects are imaged in a normal facial expression. The scheme of Chang et al., however, requires manual normalization and has to use normal facial expressions.
The work of Bronstein et al, focused on developing a representation of the facial surface, invariant to different expressions and postures of the face. A. Bronstein, M. Bronstein, and R. Kimmel, Expression invariant 3D face recognition, Audio and Video Based Biometric Person Authetication, pp. 62-69 (2003). Before using the basic idea of PCA, they calculate the geometric invariants of a face surface by using multidimensional scaling (MDS). For a discussion of MDS, see Schwartz, E. L., Shaw, A., Wolfson, E., A numerical solution to the generalized mapmaker's problem: flattening nonconvex polyhedral surfaces, IEEE Trans. PAMI, 11: 1005-1008 (1989). Bronstein et al. did not report the recognition rate, though they claimed that their algorithm can recognize the difference of twins. Although they did not discuss in detail the computation cost of their method, it appears to be high because MDS needs to calculate the geodesic distances between each pair of points on the surface, as well as the eigen decomposition of a large matrix.
Chua et al. analyzed over four expressions of each person to determine the rigid parts of the face. C., F. Han, Y. Ho, 3D Human Face Recognition Using Point Signature, 4th IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, (2000). These rigid parts are modeled by point signatures for face indexing. Their method, however, was tested on only six individuals.
Taking a different approach, Beumier et al. developed an integrated 3D face acquisition and comparison system. C. Beumier, M. Acheroy, Automatic 3D face authentication. Image and Vision Computing, 18:315-321 (2000). The structured light was used to capture the facial surface. For facial surface comparison, they abandoned feature extraction but calculated the global matching error of the facial surfaces. An Iterative Condition Mode (ICM) optimization was performed to determine the rotation and translation transform that minimizes the global matching error sampled at fifteen profiles. In order to speed up the global matching process, they further extracted the central profile with maximal protrusion (due to the nose). The central profile and a mean lateral profile were used to compare two faces in the curvature space. The main advantages of this method are its high speed and low storage needs. But as the authors pointed out, the optimization procedure used for the 3D face comparison can fail due to noise, local minima or bad initial parameters. The reported Equal Error Rate, i.e, the rate at which false acceptances (i.e., incorrectly accepting an imposter claim) and false rejects (i.e., incorrectly rejecting a valid claim) are equal (the two rates tend to be inversely rated), is 9%. So they resorted to manual refinement for surface matching. Cartoux et al. presented a similar approach to extract the symmetry profile by looking for the bilateral symmetry axis of Gaussian curvature values of the facial surface. J. Y. Cartoux, J. T. Lapreste, M. Richetin, Face authentification or recognition by profile extraction from range images, IEEE Computer Society Workshop on Interpretation of 3D Scenes, pp 194-199 (1989).
There is a need, therefore, for an improved method and system for authentication and recognition using 3D facial data that is computationally faster and more accurate in matching facial features.
Additional objects and advantages of the invention will be set forth in the description that follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the instrumentalities and combinations pointed out in the appended claims.